Add shard id field to CLUSTER SHARDS response #2568

satheeshaGowda · 2025-09-01T05:32:41Z

This change exposes an existing, persistent (in nodes.conf) unique shard Id for each shard in the cluster as part of the CLUSTER SHARDS command response.

1) 1) "slots"
   2) 1) (integer) 0
      2) (integer) 999
      3) (integer) 2001
      4) (integer) 3999
      5) (integer) 4501
      6) (integer) 5460
   3) "nodes"
   4) 1)  1) "id"
          2) "6e76043bed00e716e85035107866ea16e9a5f700"
          3) "port"
          4) (integer) 6385
          5) "ip"
          6) "127.0.0.1"
          7) "endpoint"
          8) "127.0.0.1"
          9) "role"
         10) "replica"
         11) "replication-offset"
         12) (integer) 8092
         13) "health"
         14) "online"
      2)  1) "id"
          2) "b2f8c841707b2246ec2a641c37f16e88fe0bb700"
          3) "port"
          4) (integer) 6380
          5) "ip"
          6) "127.0.0.1"
          7) "endpoint"
          8) "127.0.0.1"
          9) "role"
         10) "master"
         11) "replication-offset"
         12) (integer) 8092
         13) "health"
         14) "online"
   5) "id"
   6) "3f2a7bb7bbd5fc2a331fe9bf95f5e02bcca02430"

src/cluster_legacy.c

nilanshu-sharma

Looks good to me

src/cluster_legacy.c

satheeshaGowda · 2025-10-06T15:26:10Z

thanks for accepting the PR. The feedback was to change the response in to the following format instead.

1) 1) "slots"
   2) 1) (integer) 0
      2) (integer) 999
      3) (integer) 2001
      4) (integer) 3999
      5) (integer) 4501
      6) (integer) 5460
   3) "nodes"
   4) 1)  1) "id"
          2) "6e76043bed00e716e85035107866ea16e9a5f700"
          3) "port"
          4) (integer) 6385
          5) "ip"
          6) "127.0.0.1"
          7) "endpoint"
          8) "127.0.0.1"
          9) "role"
         10) "replica"
         11) "replication-offset"
         12) (integer) 8092
         13) "health"
         14) "online"
      2)  1) "id"
          2) "b2f8c841707b2246ec2a641c37f16e88fe0bb700"
          3) "port"
          4) (integer) 6380
          5) "ip"
          6) "127.0.0.1"
          7) "endpoint"
          8) "127.0.0.1"
          9) "role"
         10) "master"
         11) "replication-offset"
         12) (integer) 8092
         13) "health"
         14) "online"
   5) "shard-id"
   6) "3f2a7bb7bbd5fc2a331fe9bf95f5e02bcca02430"

hpatro · 2025-10-06T16:48:24Z

I think we can just do "id".

Signed-off-by: Satheesha Chattenahalli Hanume Gowda <[email protected]>

satheeshaGowda · 2025-10-06T20:25:04Z

sample cluster shards response...

127.0.0.1:16001> cluster shards
1) 1) "slots"
   2) 1) (integer) 0
      2) (integer) 5460
   3) "nodes"
   4) 1)  1) "id"
          2) "bc22cdd1ca4e799faf01129e70cc8e617e555fba"
          3) "port"
          4) (integer) 16001
          5) "ip"
          6) "127.0.0.1"
          7) "endpoint"
          8) "127.0.0.1"
          9) "role"
         10) "master"
         11) "replication-offset"
         12) (integer) 126
         13) "health"
         14) "online"
      2)  1) "id"
          2) "abc9983ecb81d5032d321c2c681e5fb6714ad720"
          3) "port"
          4) (integer) 16004
          5) "ip"
          6) "127.0.0.1"
          7) "endpoint"
          8) "127.0.0.1"
          9) "role"
         10) "replica"
         11) "replication-offset"
         12) (integer) 126
         13) "health"
         14) "online"
   5) "id"
   6) "cd7688ca959bf54a32569ccf260b26715b82a6e3"
2) 1) "slots"
   2) 1) (integer) 10923
      2) (integer) 16383
   3) "nodes"
   4) 1)  1) "id"
          2) "193bf9dee302e026232fd1e49e73db0d65344740"
          3) "port"
          4) (integer) 16003
          5) "ip"
          6) "127.0.0.1"
          7) "endpoint"
          8) "127.0.0.1"
          9) "role"
         10) "master"
         11) "replication-offset"
         12) (integer) 126
         13) "health"
         14) "online"
      2)  1) "id"
          2) "6adbdc5599872baa2498c3bf481149dec332ebed"
          3) "port"
          4) (integer) 16006
          5) "ip"
          6) "127.0.0.1"
          7) "endpoint"
          8) "127.0.0.1"
          9) "role"
         10) "replica"
         11) "replication-offset"
         12) (integer) 126
         13) "health"
         14) "online"
   5) "id"
   6) "0d77234c440a6b05c2d77d1b9f37ac63585fc747"
3) 1) "slots"
   2) 1) (integer) 5461
      2) (integer) 10922
   3) "nodes"
   4) 1)  1) "id"
          2) "2a26b2d410a99207551eda195104ceed76a37e27"
          3) "port"
          4) (integer) 16002
          5) "ip"
          6) "127.0.0.1"
          7) "endpoint"
          8) "127.0.0.1"
          9) "role"
         10) "master"
         11) "replication-offset"
         12) (integer) 126
         13) "health"
         14) "online"
      2)  1) "id"
          2) "81885765e4bcc0fb0b7a198b60e6fab4dc6be991"
          3) "port"
          4) (integer) 16005
          5) "ip"
          6) "127.0.0.1"
          7) "endpoint"
          8) "127.0.0.1"
          9) "role"
         10) "replica"
         11) "replication-offset"
         12) (integer) 126
         13) "health"
         14) "online"
   5) "id"
   6) "a96f023185ea6d7f5e9a0d6ada6f112741217e6a"
127.0.0.1:16001>

sample nodes.conf file that matches with shard id

 cat nodes-16001.conf
bc22cdd1ca4e799faf01129e70cc8e617e555fba 127.0.0.1:16001@26001,,tls-port=0,shard-id=cd7688ca959bf54a32569ccf260b26715b82a6e3 myself,master - 0 0 1 connected 0-5460
81885765e4bcc0fb0b7a198b60e6fab4dc6be991 127.0.0.1:16005@26005,,tls-port=0,shard-id=a96f023185ea6d7f5e9a0d6ada6f112741217e6a slave 2a26b2d410a99207551eda195104ceed76a37e27 0 1759782010000 2 connected
6adbdc5599872baa2498c3bf481149dec332ebed 127.0.0.1:16006@26006,,tls-port=0,shard-id=0d77234c440a6b05c2d77d1b9f37ac63585fc747 slave 193bf9dee302e026232fd1e49e73db0d65344740 0 1759782010000 3 connected
193bf9dee302e026232fd1e49e73db0d65344740 127.0.0.1:16003@26003,,tls-port=0,shard-id=0d77234c440a6b05c2d77d1b9f37ac63585fc747 master - 0 1759782010369 3 connected 10923-16383
2a26b2d410a99207551eda195104ceed76a37e27 127.0.0.1:16002@26002,,tls-port=0,shard-id=a96f023185ea6d7f5e9a0d6ada6f112741217e6a master - 0 1759782010000 2 connected 5461-10922
abc9983ecb81d5032d321c2c681e5fb6714ad720 127.0.0.1:16004@26004,,tls-port=0,shard-id=cd7688ca959bf54a32569ccf260b26715b82a6e3 slave bc22cdd1ca4e799faf01129e70cc8e617e555fba 0 1759782010000 1 connected
vars currentEpoch 6 lastVoteEpoch 0

hpatro

LGTM!

hpatro · 2025-10-06T20:30:22Z

@satheeshaGowda Could you also update cluster-shards.json file about the new field/history ?

Signed-off-by: Satheesha Chattenahalli Hanume Gowda <[email protected]>

satheeshaGowda · 2025-10-06T20:45:48Z

I think we need also update the doc site - https://github.com/valkey-io/valkey-doc/blob/main/commands/cluster-shards.md, will send that separate PR later.

Signed-off-by: Satheesha Chattenahalli Hanume Gowda <[email protected]>

hpatro

Just one more thing @satheeshaGowda, forgot to mention we auto generate/update the commands.def file. Could you perform make -j and add the diff ?

src/commands/cluster-shards.json

Signed-off-by: Satheesha Chattenahalli Hanume Gowda <[email protected]>

hpatro

@valkey-io/core-team It's good to be merged. Do we wait for the RC3 or start cherry picking changes to 9.0 release branch from now ?

zuiderkwast · 2025-10-08T02:12:07Z

~~Please update the top comment to match the implemenration before merging.~~ Done. (Keeping only the description of the change that is useful as the commit message.)

@hpatro Let's just set project to 9.0, then whoever does the release will handle it.

src/commands/cluster-shards.json

Signed-off-by: Satheesha Chattenahalli Hanume Gowda <[email protected]>

hpatro · 2025-10-08T18:12:15Z

Allowed the workflow to run the CI. It was disabled as the author is new to the project. Once that passes, I will merge it in.

codecov · 2025-10-08T18:30:34Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 72.41%. Comparing base (0646749) to head (2017bc7).
⚠️ Report is 6 commits behind head on unstable.

Additional details and impacted files

@@             Coverage Diff              @@
##           unstable    #2568      +/-   ##
============================================
+ Coverage     72.18%   72.41%   +0.22%     
============================================
  Files           128      128              
  Lines         71037    71243     +206     
============================================
+ Hits          51277    51589     +312     
+ Misses        19760    19654     -106

Files with missing lines	Coverage Δ
src/cluster_legacy.c	`87.26% <100.00%> (-0.13%)`	⬇️
src/commands.def	`100.00% <ø> (ø)`

... and 25 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

tests/cluster/tests/28-cluster-shards.tcl

madolson · 2025-10-08T21:14:19Z

We released 9.0 RC3 without this (I wasn't aware that this was pending). I think we should punt this back to 9.1.

hpatro · 2025-10-08T22:53:11Z

We released 9.0 RC3 without this (I wasn't aware that this was pending). I think we should punt this back to 9.1.

I wasn't sure as well. @zuiderkwast suggested to target it to 9.0. Do we still want to do it with GA or just punt ?

zuiderkwast · 2025-10-08T23:01:05Z

Idk. I thought we talked about it in the last meeting to be able to squeeze it in for 9.0, but I may have misunderstood. I suppose we shouldn't add new features after rc1 (or after rc2 for small ones maybe).

madolson · 2025-10-09T20:39:03Z

I suppose we shouldn't add new features after rc1 (or after rc2 for small ones maybe).

No new features after rc1, and no incremental features after rc2 is sort of my belief.

This change exposes an existing, persistent (in nodes.conf) unique shard identifier for each shard in the cluster as part of the `CLUSTER SHARDS` command response. ``` 1) 1) "slots" 2) 1) (integer) 0 2) (integer) 999 3) (integer) 2001 4) (integer) 3999 5) (integer) 4501 6) (integer) 5460 3) "nodes" 4) 1) 1) "id" 2) "6e76043bed00e716e85035107866ea16e9a5f700" 3) "port" 4) (integer) 6385 5) "ip" 6) "127.0.0.1" 7) "endpoint" 8) "127.0.0.1" 9) "role" 10) "replica" 11) "replication-offset" 12) (integer) 8092 13) "health" 14) "online" 2) 1) "id" 2) "b2f8c841707b2246ec2a641c37f16e88fe0bb700" 3) "port" 4) (integer) 6380 5) "ip" 6) "127.0.0.1" 7) "endpoint" 8) "127.0.0.1" 9) "role" 10) "master" 11) "replication-offset" 12) (integer) 8092 13) "health" 14) "online" 5) "id" 6) "3f2a7bb7bbd5fc2a331fe9bf95f5e02bcca02430" ``` --------- Signed-off-by: Satheesha Chattenahalli Hanume Gowda <[email protected]> Co-authored-by: Satheesha Chattenahalli Hanume Gowda <[email protected]>

Related to the PR valkey-io/valkey#2568 --------- Signed-off-by: Satheesha Chattenahalli Hanume Gowda <[email protected]> Signed-off-by: Harkrishn Patro <[email protected]> Co-authored-by: Satheesha Chattenahalli Hanume Gowda <[email protected]> Co-authored-by: Harkrishn Patro <[email protected]> Co-authored-by: Madelyn Olson <[email protected]>

github-actions bot assigned satheeshaGowda Sep 1, 2025

satheeshaGowda force-pushed the shard_id branch from 6827552 to e185219 Compare September 1, 2025 05:34

satheeshaGowda commented Sep 2, 2025

View reviewed changes

src/cluster_legacy.c Outdated Show resolved Hide resolved

nilanshu-sharma approved these changes Sep 3, 2025

View reviewed changes

src/cluster_legacy.c Outdated Show resolved Hide resolved

hpatro self-requested a review September 8, 2025 15:09

zuiderkwast added the major-decision-pending Major decision pending by TSC team label Sep 29, 2025

madolson added major-decision-approved Major decision approved by TSC team and removed major-decision-pending Major decision pending by TSC team labels Oct 6, 2025

Expose shard id in CLUSTER SHARDS respone

adcec3e

Signed-off-by: Satheesha Chattenahalli Hanume Gowda <[email protected]>

satheeshaGowda force-pushed the shard_id branch from e185219 to adcec3e Compare October 6, 2025 18:10

Incorporate TSC feedback

7b379b2

Signed-off-by: Satheesha Chattenahalli Hanume Gowda <[email protected]>

hpatro approved these changes Oct 6, 2025

View reviewed changes

update cluster-shards.json with shard id field

7e92b17

Signed-off-by: Satheesha Chattenahalli Hanume Gowda <[email protected]>

zuiderkwast added needs-doc-pr This change needs to update a documentation page. Remove label once doc PR is open. release-notes This issue should get a line item in the release notes labels Oct 6, 2025

Remove since field

2790628

Signed-off-by: Satheesha Chattenahalli Hanume Gowda <[email protected]>

hpatro changed the title ~~Expose shard id in CLUSTER SHARDS respone~~ Add shard id field to CLUSTER SHARDS response Oct 6, 2025

Added history field

d738ea1

Signed-off-by: Satheesha Chattenahalli Hanume Gowda <[email protected]>

satheeshaGowda mentioned this pull request Oct 7, 2025

Add shard id field to CLUSTER SHARDS response valkey-io/valkey-doc#365

Merged

hpatro approved these changes Oct 7, 2025

View reviewed changes

src/commands/cluster-shards.json Outdated Show resolved Hide resolved

updated commands.def

314ac66

Signed-off-by: Satheesha Chattenahalli Hanume Gowda <[email protected]>

hpatro approved these changes Oct 7, 2025

View reviewed changes

zuiderkwast added this to Valkey 9.0 Oct 8, 2025

enjoy-binbin approved these changes Oct 8, 2025

View reviewed changes

src/commands/cluster-shards.json Outdated Show resolved Hide resolved

Change version history to 9.0

2017bc7

Signed-off-by: Satheesha Chattenahalli Hanume Gowda <[email protected]>

hpatro merged commit 5bbbc6b into valkey-io:unstable Oct 8, 2025
53 checks passed

github-project-automation bot moved this to To be backported in Valkey 9.0 Oct 8, 2025

madolson reviewed Oct 8, 2025

View reviewed changes

tests/cluster/tests/28-cluster-shards.tcl Show resolved Hide resolved

hpatro added this to Valkey 9.1 Oct 13, 2025

hpatro removed this from Valkey 9.0 Oct 13, 2025

hpatro mentioned this pull request Oct 13, 2025

Bump CLUSTER SHARDS command update version to 9.1.0 #2729

Merged

hpatro moved this to Done in Valkey 9.1 Oct 13, 2025

Add shard id field to CLUSTER SHARDS response #2568

Add shard id field to CLUSTER SHARDS response #2568

Uh oh!

Conversation

satheeshaGowda commented Sep 1, 2025 • edited by zuiderkwast Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

nilanshu-sharma left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

satheeshaGowda commented Oct 6, 2025

Uh oh!

hpatro commented Oct 6, 2025

Uh oh!

satheeshaGowda commented Oct 6, 2025

Uh oh!

hpatro left a comment

Choose a reason for hiding this comment

Uh oh!

hpatro commented Oct 6, 2025

Uh oh!

satheeshaGowda commented Oct 6, 2025

Uh oh!

hpatro left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

hpatro left a comment

Choose a reason for hiding this comment

Uh oh!

zuiderkwast commented Oct 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

hpatro commented Oct 8, 2025

Uh oh!

codecov bot commented Oct 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Uh oh!

Uh oh!

madolson commented Oct 8, 2025

Uh oh!

hpatro commented Oct 8, 2025

Uh oh!

zuiderkwast commented Oct 8, 2025

Uh oh!

madolson commented Oct 9, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

satheeshaGowda commented Sep 1, 2025 •

edited by zuiderkwast

Loading

zuiderkwast commented Oct 8, 2025 •

edited

Loading

codecov bot commented Oct 8, 2025 •

edited

Loading